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ABSTRACT 



The School- to-Work Opportunities Act of 1994 mandates 
standards-based assessment and skills certification as an integral part of a 
comprehensive school- to-work (STW) system. An ideal STW assessment system 
would span elementary through postsecondary education and culminate in 
assessment of skills at the workplace. It would articulate in progressive and 
logical fashion and reinforce and measure individuals' proficiency with 
respect to agreed-upon standards. Skills assessment development and 
implementation must involve a partnership of educators and industry 
representatives. Starting in high school, however, industry must assume more 
responsibility and ultimately assume the lead. An ideal STW assessment and 
certification system would include assessment of the following skills in the 
following settings: academic and general workplace skills in middle school; 
academic, general workplace, and industry core skills in grade 10; academic, 
general workplace, industry core, and occupational family skills in grade 12; 
and academic, general workplace, industry core, occupational family, and 
specific occupational skills in higher education and the workplace. The ideal 
STW assessment model would recognize differences between education's and 
industry's perspectives on key assessment issues (including standards, 
accountability, legal def ensibility , multiple measures, cost, marketing, and 
portability) and bridge those differences to pave the way for successful 
industry- education partnerships. (Contains 30 references.) (MN) 
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The Ultimate School-to-Work Challenge: 

Linking Assessment in School and the Workplace 

by 

Sri Ananda, WestEd 

The school-to-work (STW) movement is an ambitious reform agenda aimed at improving 
the preparation and performance of young people for the world of work by strengthening 
the relationship between school and work. Advocates in the business and education 
communities agree about the need for coherent state and local school-to-work systems 
that span across the educational system from kindergarten to community college, or 
university, on through to the workplace. The School-to-Work Opportunities Act of 1994 
(STWOA) is a far-reaching law that establishes a national framework for the 
development of such systems, providing venture capital to underwrite the initial costs of 
planning and establishing such systems. 

With or without the direct support of the STWOA, several efforts to build school-to-work 
systems are well underway across the nation. In this process of planning and early 
implementation, it has become clear that many states and local partnerships have 
significantly underestimated what it actually takes to build a comprehensive school-to- 
work system. Several authors are beginning to address the many challenges and barriers 
faced in design and implementation of a comprehensive STW system (National 
Governors’ Association, 1996). The complexities around building sustainable education- 
industry partnerships alone are forcing many state and local partnerships to reconsider 
their designs and timelines for implementing comprehensive STW systems (Hershey, et. 
al., 1997). Given this reality, other key components of a comprehensive STW system, 
such as assessment, have yet to receive concerted attention. 

The STWOA recognizes that assessment must be an integral part of a school-to-work 
system in order to properly gauge system effectiveness and student progress. Moreover, it 
specifically calls for standards-based assessment systems that promote skill certificates. 
These certificates are to be industry-recognized, portable credentials that are issued by a 
state-approved STW program (Wills, 1997). To date, few states have even attempted to 
develop such a STW assessment system. There is a dearth of agreed upon STW 
assessment models, few resources devoted to their development, and little progress on 
this front in general. Thus, the time has come for serious attention to viable STW 
assessment models, beginning with an examination of current assessment-related issues 
and practices in education and industry. 

A comparative analysis between education and industry with respect to assessment has 
relevance to STW as well as to education and industry independent of STW. For 
example, there is currently a scarcity of standards-based models and tools to assess the 
skills of entry and expert level workers for various occupational clusters and industries. 
An examination of current assessment practices in education could help inform industry 
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on the design of standards-based instruments. Similarly, a comparative analysis could 
help education learn from industry’s experience, particularly with issues around 
performance-based assessment. 

Beyond presenting a comparative analysis of assessment in education and industry, this 
paper is intended as a “wake up” call to the field that much work remains if we are to 
successfully incorporate STW assessments and certification as part of our current efforts 
to develop comprehensive STW systems at the state and local levels. This is no easy 
undertaking; it will require intensive planning and significant resources and more debate 
on assessment forms and purposes, as opposed to the rhetoric and half measures that 
seems to dominate current treatment of this key STW component. The paper begins with 
a vision for an articulated STW assessment system. Then, it analyzes the different 
perspectives and practices of education and industry on key assessment issues that 
challenge the design and implementation of such a system. Finally, the paper identifies 
the conditions — structures, mandates, and technical considerations — that are necessary 
for successful implementation of a comprehensive STW assessment system. 

An Ideal School-to-Work Assessment System 

An ideal STW assessment system would span elementary through postsecondary 
education, culminating in assessment of skills at the workplace. It would articulate in 
progressive and logical fashion, assessing performance at key junctures in an individual’s 
education and training in order to reinforce and measure that individual’s proficiency with 
respect to agreed upon standards. In the educational arena, skills assessment 
development and implementation should always involve a partnership of educators and 
industry representatives (as well as parents and the broader community). However, 
starting in high school, industry must take on more responsibility and ultimately assume 
the lead. This is consistent with the STW and skill standards movements, both of which 
espouse that standards and assessments must be industry driven in order to have 
credibility and validity for career applications. 

A model for an ideal STW assessment system is offered below. First described elsewhere 
(Ananda & Rabinowitz, 1995), and expanded upon here, the system reinforces instruction 
and training in the knowledge and skills required at different points in an individual’s 
career preparation, reflecting the objectives of both education and industry. Figure 1 
illustrates such a system, specifying the targeted standards and skills, certification status, 
and partnerships appropriate for each point of assessment. A brief summary of this 
idealized system follows. 

Elementary and Middle School 

Targeted Standards and Skills. With career awareness instruction beginning in 
elementary school, assessment of skills that are transferable to the workplace should start 
by middle school. At this point, instruction should be on building academic foundations 
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required for full participation in society, including employment and citizenship. The 
academic curriculum should also infuse career-related themes and concepts, such as 
quality control in science education and teamwork in social studies. To reinforce such 
learning, assessment should focus not only on academic skills, but also general workplace 
readiness skills that help provide students with a solid life foundation, and are often 
excluded from the traditional academic curriculum. These general workplace skills 
address what workers must have in order to adapt to the demands of any occupation, such 
as interpersonal skills, applying technology, and personal responsibility. 

Certification Status. Assessment of the targeted skills can occur as part of the ongoing 
academic testing program and should not entail formal certifications or sanctions for 
students. In addition to traditional paper-and-pencil assessment tasks, use of assessments 
such as portfolios and teamwork exercises at this level should help reinforce both 
academic and general workplace readiness skills. 

Assessment Partners. Primary responsibility for assessment remains at the local rather 
than the state level, with educators in the lead. Although formal career-related 
certification is considerably premature at this level, community or industry groups should 
participate along with educators in development or review of the assessments. This will 
help facilitate the infusion of real-life applications with academic content. 

High School 

Two distinct levels of formal assessment and certification should be offered in high 
school, as described below. 

Grade 10 

Targeted Standards and Skills. Many states stipulate that students should complete their 
core training in key academic subjects (e.g., mathematics, science, language arts) by grade 
10. As with the middle school level, assessments at grade 10 should focus heavily on 
academic concepts. Nevertheless, students at this level should be exposed to general 
workplace readiness, such as those identified by the Secretary’s Commission on 
Achieving Necessary Skills (SCANS). SCANS (1992), which is perhaps the most well- 
known effort to identify general workplace skills, has identified three foundation skills 
(basic skills, thinking skills, personal qualities) and five key competencies (resources, 
interpersonal, information, systems, and technology). 

SCANS-like skills are often introduced in career-technical foundation or core courses, 
such as business administration core or health careers core. Such core courses also target 
industry core standards, that is, standards that apply to all or nearly all occupations in a 
particular industry. For example, a health careers core class would typically address the 
SCANS-like skill of teamwork, but in the context of the health care industry and as a 
health care core standard. As an alternative vehicle for teaching broad workplace skills, 
some states and local districts are requiring that students take a general “workplace/career 
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skills” class. This may be a less effective way of teaching general workplace skills 
because of the non-contextualized nature of such classes. At any rate, assessments at 
grade 10 should reinforce attainment of general workplace readiness and industry core 
skills. 

Certification Status. Some states are planning to issue Certificates of Initial Mastery 
(CIM) to recognize achievement in key academic subjects. These certificates are 
intended to assess and reinforce the achievement of a strong academic base, ideally by 
grade 10 and certainly by high school graduation. This process of certification shifts 
substantial responsibility for development of formal assessments from the local to the 
state level. Comprehensive STW systems should expand upon current CIM assessment 
models from their core academic base to include assessment of general workplace 
readiness skills and the introduction of career preparation focusing on a particular 
economic sector or industry (e.g., industry core standards). The primary objective of the 
expanded Certificate of Initial Mastery is to help ensure that students are exposed to 
information about broad career options early on in high school and hence do not limit 
their future options due to inadequate information or academic training. Nevertheless, 
academic preparation should still be the primary driving force behind CIM because 
academic foundations form the basis for so much subsequent learning. Again, a 
combination of traditional paper- and-pencil tasks as well as more innovative assessment 
techniques would allow the student to demonstrate mastery in different ways. 

Assessment Partners. Educators should take the lead in assessment development and 
implementation, although assessment development teams should also include parent, 
civic, and industry representation in order to secure valuable input as well as the 
necessary “buy in” for the certification process. 

Grade 12 (Exit) 

Targeted Standards and Skills. The focus should be on advanced academic skills (e.g., 
technical writing, literary analysis, geometry, biological sciences) that are reinforced, 
where appropriate, in the context of a career specialization. Many recent school-to-work 
models stipulate that academic training after grade 10 should be integrated within the 
context of an occupational family (e.g., agriculture science, food services) or career 
cluster. For example, career academies built around industries ranging from health to 
electronics represent a specialized attempt to combine academic and career training into 
meaningful student-based experiential learning (Stem, Raby, and Dayton, 1992; 
Linnehan, 1996). 

In general, high school exit assessments should focus on broad career preparation within 
or across industries (i.e., the occupational family), including strong emphases on self- 
directed learning and academic achievement. This broader preparation will better serve 
youth in the long run because it will give them the flexibility needed to adjust to the 
changing workplace and accompanying changes in specific job demands and 
requirements. For a high-school student, job-specific training may be too narrow and 
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may limit the student’s later options. With a few notable exceptions, such as auto 
mechanics, occupation-specific training and assessment should be the focus of higher 
education and the workplace, rather than high school. 

Certification Status. Whereas grade 10 certification focuses on academic foundations, 
general workplace readiness, and industry core skills, at or around grade 12 students 
could pursue a more advanced level of certification that demonstrates mastery of a more 
specialized level of academic and technical knowledge and skill relevant to an 
occupational family. States such as Oregon refer to this level of high school certification 
as Certificates of Advanced Mastery (CAM). While Oregon’s CAM currently focuses 
more on advanced academic rather than career specialization skills, the concept of the 
expanded Certificate of Advanced Mastery presented here would actively support both 
options. That is, Certificates of Advanced Mastery could be offered in humanities (e.g., 
for college-bound liberal arts majors) or occupational families, such as graphic design, 
bioscience, or electronics. In Oregon, development of endorsement credentials are 
currently underway, providing an option for students to be recognized for in-depth career- 
related study. 

The Certificate of Advanced Mastery may allow for greater use of performance-based 
assessment (e.g., projects, portfolios, demonstrations) than the Certificate of Initial 
Mastery. As compared to the more general Certificate of Initial Mastery, fewer students 
may be prepared to pursue Certificate of Advanced Mastery in career-related fields. 
Lower numbers can mean more manageable assessment administration and scoring 
demands, thereby making performance-based assessment methods a more feasible option. 
Moreover, the more specialized preparation within an occupational family provides 
greater opportunities for performance-based assessment. For example, students at this 
level have more career-related content knowledge to develop long-term projects (e.g., 
patient case studies in health careers; comprehensive marketing plans for business 
administration) that can serve as assessment tasks. Finally, students’ advanced training 
often includes paid or unpaid on-the-job experience which also lends itself to more 
“hands on” performance assessment (e.g., job/ work samples). 

Assessment Partners. As compared to earlier grade levels, the career specialization at 
grade 12 suggests that industry must take a more active role in assessment development 
and implementation. For example, employers and workers could help design and 
evaluate more “authentic” assessment methods that reflect workplace practices and needs, 
such as projects and work samples. At this point, there must be full buy in from both the 
education and industry partners. In this way assessments can be closely tied with industry 
expectations but still have fidelity to classroom curriculum and practices. In addition, 
parents and the broader community should be involved in the assessment development 
and implementation process. 
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Higher Education 



Targeted Standards and Skills. A traditional focus for community college curriculum is 
on occupational specific training (e.g., nursing assistant, welding, drafting). However, 
training at this level is also broadening to include fundamental preparation in a specific 
industry (core curriculum) or occupational family to help prepare students for horizontal 
and vertical movements in their targeted career areas. In addition to high level academic 
skills, assessments at this level should encompass general workplace readiness, 
occupational family, and occupational specific skills, as appropriate. 

With the notable exception of professional schools (e.g., education, engineering, 
business), the four-year colleges and universities have traditionally played a less active 
role in articulated career preparation. Career preparation is often perceived as 
inconsistent or even at odds with a liberal arts curriculum. However, the school-to-work 
movement is beginning to make important in-roads in involving such institutions in state 
and local school-to-work partnerships. Boeing Company, the number one aircraft 
manufacturer world wide, provides an example. Its STW training program in 
Washington state expands the traditional tech-prep concept (i.e., an articulated high 
school/community college program) to include an option for transfer to a four- year 
institution, Western Washington University (Packer, et. al., 1996). 

Certification Status. Higher education should directly prepare students for various types 
of occupational certification, including technical (e.g., radiological technician) and 
professional (e.g., accounting). Certification should involve multiple modes of 
assessment, for all the reasons alluded to previously. 

Assessment Partners. At this level, the links between education and industry need to be 
strongest because this is the major point of transition into careers for an increasing 
number of individuals. For the first time in the educational/training process, industry 
should drive the assessment development and implementation process, with full 
participation of educators and other stake holders. 

Workplace Skill Assessment 

Targeted Standards and Skills. Success in the workplace requires proficiency on a full 
range of skills. At this level, there is little need to distinguish between academic, industry 
core, or occupational family skills; all contribute to success in a specific occupation. 

Certification Status. Entry into many occupations is regulated through certification and 
licensure processes. The assessments used for certification and licensure typically are 
called for and supported by professional associations, as well as the state and federal 
governments. Many well-established certification programs, such as the National 
Institute for Automotive Service Excellence and the American Institute of Certified 
Public Accountants, rely heavily on multiple-choice assessment because of the many 
advantages associated with this assessment methodology. Multiple-choice items afford 
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broad coverage of content over a relatively short amount of time, have a strong empirical 
history with demonstrated technical quality, and are easy and relatively inexpensive to 
administer. These qualities are very appealing to large-scale certification efforts. In 
contrast, many programs have found performance-based assessments to be preferable but 
cost prohibitive (Wills, 1994). Several existing certification systems, such as the 
Certified Public Accountant’s exam (maintained by the American Institute of Certified 
Public Accountants) augment their multiple-choice components with essay or selected 
performance-based assessment tasks. 

Job entry is but one application of assessment for the workplace. For example, training or 
retraining is being offered at the workplace for new and incumbent workers. Assessment 
at end of training helps to gauge the effectiveness of the training as well as the skill level 
of the individual worker. As work site training is often short term in nature (e.g., two days 
to six months), incorporation of time-intensive assessment tasks, such as portfolios, may 
not be feasible. Instead, on-demand or “on the spot” assessments that can be 
administered in a limited period of time may be more appropriate. This could include 
multiple-choice examinations, essay type examinations, or direct observations of work. 

Likewise, workers in any industry are subject to periodic review, typically by their 
immediate supervisors and sometimes by their peers. Performance review should be 
based on explicit criteria of which both the supervisor and employee are aware. These 
criteria might include company standards or more broadly validated standards, such as the 
national industry skill standards that have been developed and validated by various 
industries with support from the U.S. Departments of Labor and Education as well as the 
National Skill Standards Board (National Skill Standards Board, 1996). Finally, 
assessment of an individual’s skills can be used as evidence to help “make the case” for 
an individual’s career advancement or promotion. For both performance review and 
promotion, a collection of an individual’s work samples, such as a portfolio, could 
provide a useful tool for performance evaluation. 

Assessment Partners. For all workplace assessment applications, industry clearly needs 
to be the driver in developing and administering such programs. However, education 
should participate in certain applications, such as certification for job entry. Involving 
educators would allow them to better understand the skills coveted by industry. They can 
then bring back this knowledge to help ensure that what occurs during the formal 
education process is better linked to workplace needs. 

Key Differences in Perspectives and Practices between Education and 

Industry 

The ideal STW assessment system described above envisions students as prospective 
workers and workers as ongoing students (WestEd, 1998). Within this system, 
assessments can measure progress, support the developmental nature of education and 
training, and provide an accurate picture of workforce readiness. It can also help close 
the gap between education and industry contexts in terms of their goals and how 
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assessments are used to support these goals. However, in order to achieve a fully 
articulated STW assessment system, we need to better understand and take into account 
the similarities and differences between education and industry with respect to existing 
assessment practices and perspectives. Interestingly, what may on the surface appear to 
be similarities play out differently in education versus industry. For example, while there 
is agreement that assessments should be grounded in high standards, there are real 
differences in how the two define high standards. Moreover, while both education and 
industry are driven by legal defensibility concerns, what constitutes evidence of validity 
in support of legal defensibility differs in subtle ways between the two. This section 
explores the differences in perspectives and practices of education and industry on key 
assessment issues, including standards, accountability, legal defensibility, multiple 
measures, cost, marketing, and portability. 

Focus on Standards 

The movement is well underway in both education and industry to set high performance 
expectations for students and workers. This has entailed development of systems of (1) 
academic standards specifying what students should know and be able to do in 
kindergarten through grade 12 (or even 16) and (2) industry skill standards that specify 
the knowledge and skills needed for successful preparation and performance of workers 
in certain occupations and industries. Both systems represent an expansion of older 
competency-based standards systems. The key notion underlying these newer standards- 
based reform movements is that setting high expectations for all students or workers is a 
fundamental step to improving their achievement levels. That is, if you provide support 
for individuals to reach these expectations and then hold them accountable, they will 
perform. 

While there is a strong sense among both educators and industry representatives that 
assessments should be linked to challenging standards, there is no consensus across 
education and industry lines about what constitutes appropriately challenging standards. 
On the education side, the focus is on setting high standards in academic content areas. 
Several sets of national standards in various content areas have been released in recent 
years, including mathematics, history, geography, civics, social studies, science, 
English/language arts, and foreign languages. Some of these efforts have generated lively 
debate. For example, an early version of the national history standards developed by the 
National Center for History in the Schools based at the University of California in Los 
Angeles, met with widespread criticism in 1995 for downplaying the role of Western 
civilization. The revised version of the standards released in 1996 met with much more 
positive reviews in the educational and broader communities. 

Regardless of the controversial nature of some of the national academic standards efforts, 
on the whole, the recently developed standards reflect more problem-solving and analysis, 
and hence are more challenging than those produced in previous times. For example, the 
widespread emphasis on minimum competency testing in our nation’s schools in the 
1970s and 1980s demonstrated our priority at the time of students’ achieving minimally 
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acceptable levels of skills or standards. While the emphasis has now shifted to high 
levels of standards, the standards reform movement in the education community still 
focuses the lion’s share of attention on academic content areas. Applied learning, service 
learning, and career preparation are only recently beginning to receive some attention 
(New Standards, 1995; Council of Chief State School Officers, 1997). 

In contrast, occupation- or industry-specific skill standards drive assessment for the work 
place. Industry skill standards are industry-validated sets of standards making explicit 
what knowledge, skills, and performance levels workers need to achieve in order to be 
successful in a particular job, occupational family, or industry. Over the last four years, 
efforts to develop national skill standards for more than 20 industries have received 
support from the U.S. Departments of Education and Labor as well as the National Skill 
Standards Board 1 . 

While industry is beginning to embrace skill standards, to date, the reaction by educators 
has been mixed. Skill standards are considered by a number of educators to inadequately 
reflect high level academic knowledge and skills. This was a prevalent viewpoint among 
educators as expressed at a key meeting of academic and industry skill standards 
developers and users in Harriman, New York in summer 1996 (Bailey, 1997). At this 
meeting, it became clear that many educators view skill standards in a secondary position 
to academic standards, that is, as a tool for reinforcing academic skills. For example, 
measurement tasks or problems in manufacturing can help reinforce mathematics 
achievement by providing hands-on or “real life” applications. Given this perspective, 
the more academically challenging the skill standards, the better. Fidelity to the needs of 
the workplace is not the primary consideration. 

However, from the industry viewpoint, skill standards must reflect directly what is needed 
in the work place. Although employers want their workers to meet challenging standards, 
they are also legally bound to using standards that reflect actual job requirements. For 
example, if the job in question does not currently require high levels of written 
communication, problem solving, science, or mathematics skills, then workers cannot be 
held to those standards for job entry. Industry-driven standards must be grounded in 
current practice and cannot embody lofty goals that are not supported by practice, as 
advocated by some educators. To embrace standards or assessments that go beyond 
current practice is to leave an employer vulnerable to legal challenge (see discussion on 
legal defensibility below). 

However, what constitutes current practice in many workplaces is changing. Increasing 
numbers of companies are ascribing to the model of a “high performance workplace,” 
demanding new skills and abilities of workers. Companies that follow a high 
performance model share many of the following characteristics: strong goal orientation, 
emphasis on quality, performance, and continuous improvement, decentralization of 
authority and responsibility, more direct worker control over job outcomes, and more 

1 The 1994 National Skill Standards Act charged the National Skill Standards Board with “stimulating the 
development and adoption of a voluntary national system of skill standards.” 
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comprehensive formal and informal education programs (Nash & Korte, 1997). The 
emergence of the high performance workplace will help set the stage for more 
challenging industry skill standards. 

Accountability 

In both the educational and industry arenas, assessments serve an accountability function, 
that is, to ensure that goals are met. For education, student assessment provides a 
yardstick by which to hold schools, teachers, and students accountable for achieving 
agreed upon levels of student achievement or performance. In the workplace, the 
employer is accountable to the client or consumer for service or product quality. 
Employee performance and productivity must support this accountability. Thus, 
assessments of an individual employee’s performance provide useful tools for hiring, 
performance review, and promotion decisions. 

While accountability is key to assessment in both arenas, the nature of that accountability 
is very different, driving assessment in distinct ways. For example, at the elementary and 
middle school levels, the primary role of assessments is to help ensure that students are 
ready for the next level of education. Hence, the focus is upward, or towards the future, 
measuring progress as students climb the educational ladder. At the same time, 
educational assessments are also supposed to serve a general diagnostic function, such 
that results can be used to signal the need for possible targeted help and remediation for 
specific individuals so that they may progress to the next levels of education. 

In contrast, assessments used for the workplace — particularly occupational certification- 
are intended to ensure that workers possess the skills necessary to do the job. They serve 
to protect the employers’ and the general public interest by screening out unqualified 
workers. As such, the focus of accountability is on the present, rather than on the future: 
Does this individual currently possess the skills necessary for success on the job? 
Moreover, the diagnostic function of assessments does not figure as strongly into the 
accountability equation as it does for educational assessment. Whereas evaluative 
reporting of assessment results is demanded by schools and parents on the education side, 
employers are not making such demands of occupational certification. 

The differences in accountability functions are reflected in how education and industry 
view certification. For industry, occupational certification has traditionally been defined 
as a process of demonstrating knowledge and skill in a given specialization area for the 
purpose of maintaining minimal ethical and performance standards. Thus, certification in 
industry connotes minimal standards, whereas certification in education connotes higher 
standards (Mulkey & O’Neil, 1997). 

Legal Defensibility 

Assessments in education and industry are subject to an array of legal challenges. This is 
particularly true when high stakes are attached to assessment results, such as whether or 
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not to certify a worker for occupational competence or graduate a high school student. 

The types of legal challenges high stakes educational and workplace skill assessments 
face are comparable in many ways. For example, substantially lower performance on 
assessments by historically disadvantaged groups as compared with others frequently 
form the basis of challenges to which high stakes assessments are vulnerable. 

In educational and workplace skill assessment, group differences in performance in and 
of themselves do not signify unfair or biased assessment tasks. Standard practice now 
includes use of statistical and judgmental methods to further explore possible bias against 
ethnic groups or by gender (e.g., statistical comparisons of group performance that control 
for ability level; review of assessments by groups of content experts with substantial 
representation from minority groups). On the industry side, claims of “disparate impact” 
or unfair negative effects on a specific group (e.g., minorities, women) need only 
demonstrate that the particular group has been significantly disadvantaged by an 
assessment or other employment practice. A substantial body of literature on such group 
differences as well as the question of “test fairness” has emerged from assessment 
activities conducted in the military setting (Baker & O’Neil, 1994). 

Legal defensibility also requires evidence of validity~that the specific inferences made 
from assessment scores are appropriate, useful, and meaningful. Criteria for validity have 
been developed by the joint committee of the American Educational Research 
Association, American Psychological Association, and National Council on Measurement 
in Education (1985). These criteria are currently undergoing extensive updating and 
rewriting. 

Currently, both education and industry focus on construct and content-related, aspects of 
validity for purposes of legal defensibility; Are score inferences consistent with 
theoretical concepts that account for performance and relationships with other variables? 
Is the content assessed representative of the targeted behavioral domain? However, 
courts have looked for somewhat different evidence of validity for high school graduation 
tests as compared with licensure or employment tests. For example, the need to 
demonstrate opportunity to learn and curricular validity distinguish high school 
graduation tests from tests used for employment purposes. The landmark Debra P. v. 
Turlington case in Florida challenging the state’s minimum competency requirement for 
high school graduation set the stage for both requirements (Phillips, 1995). That is, the 
decision in that case set forth that group differences in student performance likely reflect 
disparities in students’ opportunities to learn the content covered by the test and set the 
requirement that diploma tests demonstrate curriculum relevance and representativeness. 

In educational circles, validity investigations are now shifting to include emphasis on 
consequences of assessment on practice. This is referred to as consequential validity. To 
date, there is little attention to consequential validity in industry-driven assessment and 
certification. 
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While demonstrating content validity is crucial for both educational and employment 
tests, the courts have been particularly insistent that assessments used for employment 
purposes demonstrate a logical and empirical relationship between job requirements and 
assessed knowledge and skills. As Knapp (1995, p. 18) argues: “At this time, content 
validity is the method of choice for documenting the validity of credentialing 
examinations.” Thus, the content domain to be covered by a certification test should be 
explicated and justified in terms of the importance of that content for competent 
performance. Legal guidelines and accepted standards of practice point to job analysis as 
the means for generating detailed information about the tasks job incumbents perform and 
the knowledge and skills required for competent performance. 

Multiple Measures 

Assessments in both industry and education are moving towards inclusion of multiple 
measures. The accumulation of multiple forms of assessment evidence to reach a 
decision, such as certification, is referred to as triangulation. Triangulation allows for 
assessment of skills through multiple and complementary means, thereby helping to 
increase the reliability of the resultant score and any decisions made from this score 
(Mathison, 1988). Using a variety of assessment components and methods allows 
examinees to demonstrate competency in different ways, capitalizing on each individual’s 
areas of strengths, and satisfying equity and other legal concerns described above. While 
both industry and education are making concerted efforts to include innovative 
assessment tasks and methodologies, neither is abandoning multiple-choice items as an 
integral part of an overall assessment system. Both recognize that multiple-choice 
assessment tasks have a history of careful empirical development and are an efficient way 
to cover a large number of standards. 

Where industry and education differ is in the types of non multiple-choice assessment 
tasks on which they are currently focusing. For example, portfolios, or purposeful 
collections of student work, enjoy strong popularity as an alternative assessment method 
in education circles. With the exception of some of the professions (e.g., architecture, 
teaching), use of portfolios is not as widespread in industry. In fact, employers have 
expressed reservations about the feasibility of portfolios as a tool for many workplace 
assessment purposes, such as screening of potential applications (WestEd, 1998). A 
specific concern is that employers may not have the time to screen hundreds or thousands 
of portfolios to fill a specific position. Instead, employers are more interested in work 
samples, simulations, and observations as more discrete and efficient non-multiple choice 
assessment tasks. 

Whether for education or industry, the multiple measures approach to assessment presents 
important technical choice points and considerations. For example, how should 
assessment results be reported, as a single score dr profile? What is the most appropriate 
weighting scheme for the various assessment components? How should scores be 
equated? 
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Cost 



Two key cost questions related to assessment are: Who pays? and How much? In the 
education arena for the K-12 system, costs for assessment development, implementation, 
and administration are typically the joint responsibility of the state and local districts. 
Particularly for certification issued at the state level via the schools (such as the 
Certificates of Initial and Advanced Mastery), the costs of assessment leading to such 
decisions should be the primary responsibility of the state (Ananda & Rabinowitz, 1995). 
Students do not and should not be expected to bear the brunt of costs for assessment and 
certification because this may limit some students’ access to certification. A notable 
exception includes the fees charged to students for assessments such as those used for 
college entry, like the Scholastic Achievement Test (SAT), or those used to award college 
credit, like the Advanced Placement examinations. 

Assessments for the workplace are currently supported by specific industries, professional 
organizations, employers, and individual candidates. For example, professional and trade 
organizations support a number of certification programs that then, in turn, charge fees to 
individuals interested in becoming certified. At the work site, many employers use their 
own resources to develop and administer tests to screen potential candidates for vacant 
positions. 

The actual costs for assessments vary widely and are hard to gauge. Costs depend on the 
number of test takers, number, nature, and length of responses to be evaluated, the 
number of scorers, the number of topic areas to cover, the number and types of reports to 
be produced, and the inclusion of preparation and support materials. Moreover, some of 
the costs related to assessments are absorbed into other ongoing efforts, making them 
hard to estimate accurately. For example, the costs associated with school-based 
assessments, such as teachers’ time spent in preparation, administration, and scoring are 
typically incorporated into schools’ regular operations and not priced out separately (Linn 
& Herman, 1997). Such “hidden” teacher costs are also associated with the cost of “lost” 
teaching time. 

What is clear is that assessments that require constructed responses are typically costlier 
than multiple choice tests. While the latter can be machine scored, constructed response 
items must be judged by individual raters (e.g., employers, teachers, content experts). 
Administration of a machine-scorable test at a school site may cost between $5 - $10 per 
individual, depending on the volume of tests and the scoring services required. Some 
estimates are even lower. For example, the prorated cost of one class period of multiple- 
choice science assessment was reported as $1 per student (Comfort, 1995). In contrast, 
student assessments that involve short answers and extended written responses can cost 
more than two to three times as much, while those including hands-on tasks can range 
from $30 to $70 per student (Linn & Herman, 1997; Stecher et. al., 1997). This does not 
include operational costs for schools, districts, or states. On the industry side, candidate 
fees for becoming certified (for example, travel agents, insurance brokers, teachers, 
psychologists) range from $75 to $2,000 (Knapp, 1995). 




14 

18 



A promising avenue for limiting assessment costs in both the industry and education 
arenas is the computer-based approach. Computer-based administration, scoring, and 
reporting of performance assessments may be the most time and cost-effective way of 
using open-ended and innovative assessment methods. For example, the Center for the 
Research on Evaluation, Standards, and Student Testing (CRESST) has done a substantial 
amount of work in investigating the feasibility of using networked computers to capture 
and measure team competencies for individual students and teams (O’Neil, Chung, & 
Brown, 1997). 

As we begin to build articulated STW assessment systems, attention will need to focus on 
avenues for financing the new forms of assessment. A commonly held view is that the 
employers should shoulder more of the responsibility for this type of assessment because 
they would have a better prepared workforce, which would save on training and retraining 
costs. However, the reaction by the employer community to date on financial support for 
STW and work-related certification has been mixed (Wills, 1997). 

Public Engagement and Marketing 

Securing broad-based support and involvement is key to the success of high-stakes 
assessments in education and industry. In the education arena, “public engagement” -- 
the process of informing the public and securing buy in for assessments — is more critical 
now than in the past, particularly as more and more states and local districts begin 
incorporating innovative assessment tasks as part of the overall student assessment 
system. A number of states, like California, have underestimated the time and effort 
required to engage the public and reach consensus around what is important for all 
students to learn and demonstrate. A circumventing of the process there led to charges of 
elitism, secrecy, and failure of the assessments to reflect community values. This public 
relations fiasco a few years ago ultimately contributed to the demise of California’s 
statewide student assessment program, the California Learning Assessment System (Kirst 
& Mazzeo, 1996). Ironically, this need for greater public engagement emerges at a time 
when assessment development timelines seem to be getting shorter and shorter. 

For workplace skill assessment, employers to a large extent are in the driver’s seat. They 
must be convinced of the direct effect of assessments on the bottom line. That is, does 
the use of skill assessments help to predict or identify the most productive employees? 
Will the investment in assessments ultimately result in reduced costs or increased 
productivity for the company? 

In summary, for both education and industry public engagement or marketing should be 
recognized early in the planning process as a critical activity in the development and 
implementation of assessment system. 
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Portability of Certification 

Historically, the issue of portability of certificates or credentials has been of greater 
concern in industry than in education. (However, one could argue that current efforts to 
raise academic standards and the assessments on which they are based represent an 
attempt by policymakers and the public to increase the portability of the high school 
diploma.) For the most part, the business community supports centralized, nationalized 
skill standards and assessment systems (Wills, 1997). Having skill credentials that are 
portable across geographic boundaries makes the screening and hiring process much more 
efficient for employers. For workers in a given occupation or profession, portable 
certification means widespread recognition of their skill level. Furthermore, public 
interest and safety often dictates the need for portable skill credentials, particularly in 
service and health-related industries. 

In contrast, the question of portability of student certification in education brings up the 
issue of local versus state or national control. Education and training in the U.S. is highly 
decentralized and does not lend itself readily to a top-down approach typically required to 
ensure portability. States and localities currently have the power to determine for 
themselves what students should know and be able to do. Hence, the status quo is 
characterized by a myriad of state and local student assessment systems. However, if the 
goal is for STW certification to be portable across states and regions, some degree of 
local flexibility may need to be sacrificed. Moreover, models exist for national 
assessment efforts, such as the National Assessment for Educational Progress (NAEP) 
which as been regularly conducting assessments of the nation’s students since 1969. 
However, NAEP does not represent a fully developed model for STW certification 
since it does not report individual scores. 

Summary of Comparison 

The comparison of education and work place skill assessment with respect to key 
assessment issues is summarized in Table 1. Because the two worlds of assessment must 
be connected in a comprehensive STW assessment, the similarities and differences 
between the two must be considered. If not, the differences between education and the 
workplace can certainly threaten the development of a viable STW assessment system. 
The next section speaks to the conditions that must exist to help bridge these differences 
and pave the way for successful implementation of a comprehensive STW assessment 
system. 
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TABLE 1 

Comparison of Education vs 
Workplace Skill Assessment 



Standards 


School Based 
Academic 


Work Based 
Industry 


Accountability 


Future oriented: 

Ensure readiness for next 
educational level 


Present oriented: 
Screen out unqualified; 
Select qualified 


Multiple 

Measures 


Multiple-choice assessment 
Open-ended response tasks 
Portfolios 


Multiple-choice assessment 
Open-ended response tasks 
Work samples and simulations 


Validity/ 

Legal Defensibility 
Focus 


Opportunity to learn 
Curricular alignment 
Consequential validity 
Equity 


Job relevance 
Predictive validity 
Equity 


Policy 

Driver 


Public Support 


Market Support 


Who Pays? 


Public 


Industry 

Individual 


Focus on 

Portable Credentials 


No 


Yes 



Conditions for Successful Implementation of a STW Assessment System 

Successful implementation of a STW assessment system entails significant changes to the 
status quo. The conditions needed to bring about and sustain such changes include: 
structural changes, mandates to support STW assessment, and technical support. 

Structural Changes 

Although seemingly self-evident, the major structural prerequisite for an effective STW 
assessment system is a sustainable, comprehensive STW system. In turn, a STW 
assessment system would help strengthen and expand an overall STW system. 
Unfortunately, in many states the infrastructure to support a STW assessment system is 
not yet in place. Specifically, a STW assessment system needs the structural support 
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afforded by several key STW programmatic features, including an education-industry 
partnership, professional development opportunities for teachers and industry 
representatives, integrated curriculum, as well as other school-based and work-based 
components. 

The education-industry partnership must include policy- and decision-makers from both 
sectors. Only with commitment from the highest levels is it possible to make the 
significant structural changes needed to support a STW assessment system as well as 
other key components of a comprehensive overall STW system. The education-industry 
partnership should also encompass practitioners, providing a forum for teachers and 
industry representatives to develop assessment tasks collaboratively, as alluded to earlier 
in this paper. Participating teachers and industry representatives (e.g., work-site 
supervisors of students in paid and unpaid internships) would need substantial 
professional development to prepare them for designing assessment tasks, administering 
these tasks, and ensuring alignment of classroom/work-based activities with the 
assessment tasks. Finally, integrated curriculum that blends academic and professional- 
technical education and incorporates career-related learning must be in place so that 
students have the opportunity to learn the academic and professional-technical skills the 
assessment tasks are designed to measure. For a STW assessment system to be valid and 
legally defensible, it should be aligned to an integrated curriculum that exposes students 
and work candidates to the skills that they will be held accountable for in the workplace. 

Mandates to Support STW Assessment 

The STW programmatic features described above represent significant changes to 
traditional delivery of education and training. Indeed, STW is not business as usual. The 
substantive changes needed to support an over STW system, in general, and a STW 
assessment system, in particular, require strong incentives, including explicit mandates. 
Without such mandates, change can be slow, or worse, indefinitely stalled. California’s 
Career-Technical Assessment Program (C-TAP) provides an example of an innovative 
STW assessment model that is available to California high schools to assess and certify 
vocational students for proficiency on important career-technical and related academic 
standards (Rabinowitz, 1997). Use of this system in the state remains low to a large 
extent because there has not been a state or local mandate to incorporate the system as 
part of an overall STW program or system. This situation is likely to change since 
California was recently awarded a STW implementation grant under the School-to-Work 
Opportunities Act (STWOA). The grant should provide a necessary lever for California 
to support STW assessment leading to formal skill certificates. 

There are other mandates besides those afforded by the STWOA that should drive 
development of a viable STW assessment system, such as the aforementioned Certificates 
of Initial and Advanced Mastery (CIM/CAM). First Called for by the Oregon Educational 
Act for the 21 st Century (1991), Oregon’s State Board of Education 
adopted the new standards for CIM and CAM in 1996. Requirements for the Certificate 
of Advanced Mastery, in particular, represent the fiat for development of a STW 
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assessment system by calling for achievement of grade 12 state performance standards in 
career-related learning standards (e.g., personal management, problem solving, teamwork, 
etc.) and academic content areas (e.g., English, mathematics, science, social sciences, 
second language) as well as participation in a broad grouping of related careers (Arts and 
Communications, Business and Management, Health Services, Human Resources, 
Industrial and Engineering Systems, Natural Resource Systems). Further driving the 
development of a STW assessment system is the optional endorsement credential 
currently under development that will be awarded to students who complete in-depth 
career-related study (Oregon Department of Education, 1997). 

Industry mandates could also help “jump start” the adoption of a STW assessment 
system. If industry formally recognizes STW skill certificates and sends out the word that 
they intend to hire those with such certificates, assessment would become a priority for 
the emerging STW partnerships. Finally, formal adoption by a state or locality of the 
national skill standards for one or more industries could also serve as an effective 
incentive or mandate for development of a STW assessment system. 

Technical Support for Change 

As with any new or innovative assessment system, technical issues often threaten full 
implementation. Thus, low reliability and generalizability are often advanced as technical 
flaws in new predominantly performance-based assessment systems (Koretz et. al., 1992; 
Stecher et. al., 1997). Likewise, combining scores across varied assessment tasks to 
make a certification decision as well as equating of evolving assessments across years 
pose challenges to the technical adequacy of a STW assessment system. 

Although the technical challenges are real, there is evidence that many technical issues 
can be resolved through structural changes. For example, several recent studies show that 
reliability of performance-based assessments increase to acceptable levels through 
adequate training of scorers on well-defined rubrics (Baker, 1992; Shavelson, Baxter, and 
Gao, 1993). The experience in Kentucky with its statewide writing portfolio has shown 
significant improvements in scorer reliability through establishment of a statewide 
network of cluster leaders who work with local teacher/scorers. Kentucky’s experience 
clearly demonstrates that innovative assessment systems, (such as a STW assessment 
system), entail increased technical requirements, and hence, require concerted 
professional development of teachers and industry representatives to meet these 
challenges. 

Concluding Remarks 

Clearly, much work needs to be done to pave the way for the development of 
comprehensive STW assessment systems. Nonetheless, efforts have begun. Although 
still very much in its infancy stage, the groundwork is being laid to develop STW 
assessments and portable skill certificates. At the national level, the National Skill 
Standards Board (NSSB) is providing support to national partnerships that are 
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collaborating on standards and assessment work. One of these efforts, the National Skill 
Standards and Assessment Collaborative (NSSAC) is a partnership among four diverse 
industries (electronics, health care, human services, and retail) to develop and pilot-test 
assessments that apply across industries, including work place and educational settings. 
Preliminary findings from this project indicate that written scenarios and portfolios hold 
promise as assessment tools that are applicable across various industries and educational 
settings. At the same time, findings suggest significant differences in conceptions of 
what constitutes viable assessment methodologies. For example, employers participating 
in this study expressed strong preferences for written scenarios that are tailored to specific 
industry contexts and jobs in contrast to many educators who favored less occupationally 
specialized scenarios. Moreover, employers’ general stance towards portfolios is “slim is 
better” as compared with educators who tend to work with more expansive portfolios. 
Many of the observations described in this paper grew out of this particular effort which 
was directed by the present author. 

Among the other pioneering efforts to develop STW assessment systems are multiple- 
state consortia supported through the NSSB’s and National School-to-Work Office’s 
“Building Linkages” initiative to establish partnerships that link academic and industry 
skill standards, develop common career pathways, and begin design of portable skill 
certificates. Three “Building Linkages” projects are underway, one focusing on health 
care, another on manufacturing, and a third on business and management. 

As these national and multi-state efforts begin to make strides alongside individual state 
and local STW efforts, it becomes increasingly clear that significant resources are needed 
to make structural changes, establish clear mandates, and provide the technical support to 
enable successful implementation of STW assessment systems. Above all, however, 
education-industry partnerships need the will to make it happen. 
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